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^ , 1 Introduction and main results 

> , 

'nI" ! Let V^he a real vector space with a scalar product, and W C 0(Vk) be a finite group generated 

by reflections. In this paper we construct a family of M^-invariants (which we refer to as canonical 
^ ■ invariants) in S{V), where F = C (8) Vr, by means of Dunkl operators (see [8]). These canonical 

, invariants form a basis in S{V)^ (depending on a continuous parameter c) and, as such, include 

both the c-elementary and c-quasiharmonic invariants introduced in our earlier paper [2]. Using 
this technique, we prove that for W = Sn the c-elementary invariants are deformations of the 
elementary symmetric polynomials in the vicinity of c = 1/n. 

Dunkl operators Vy, y (zV*, are differential-difference operators first introduced by Charles 
^ ' Dunkl in [8] and given (for any W) by: 



V. = 9,-^c(.)^(l-.), 

s£S ^ 

where S is the set of all reflections in W, c : S — > C is a M/^-invariant function on S, and ag (zV 
is the root of the reflection s. In particular, for W = Sn, 



l<i<j<n 



where Sij G Sn is the transposition switching Xi and xj. 

The remarkable result by Charles Dunkl that all V^^ commute allows to define the opera- 
tors Vp, p e S{V*), by Vp+q = Vp + Vg, Vpg = VpVg for all p,qe S{V*). 

In what follows we will mostly think of c as a formal parameter in the affine space 
where S/W is the set of VF-orbits in 5. Using the notation Sc{V) = C(c) (g) S{V), consider each 



*This paper is a contribution to the Special Issue on Dunkl Operators and Related Topics. The full collection 
is available at |http:/ /www.emis.de/journals/SIGMA/DunkLoperators.html| 
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Dunkl operator Vp as a C-linear map S{V) — > Sc{V) (or, extending scalars, as a C(c)-linear 
endomorphism of Sc{V)). 

Thus, the association p ^ Vp defines an action of Sc{V*) on Sc{V) by differential-difference 
operators. In turn, this action and the isomorphism V = V* (hence Sc{V*) — Sc{V)) given by 
the scalar product define the bilinear form (•, ■)c : SdV) x Sc{V) — > C(c) by 

(/,5)c = [V/(5)]o, (1.1) 

for all f,g & Sc{V) where x i— > [x]q is the constant term projection SciV) — > C(c). Clearly, 
Ui9)c = if /, 5 are homogeneous and deg/ ^ deg(7. 

The form (11. ip is symmetric and its specialization at generic c : 5 ^ C and c = is nonde- 
generate. Understanding the values of c when the specialization of the form is degenerate and 
the structure of the radical is crucial for the study of representations of the rational Cherednik 
algebra Hc{W) (see e.g. PHIl]). 

A classical Chevalley theorem [6] says that the algebra S{yY^ of H^-invariants in S(y) is iso- 
morphic to the algebra of polynomials C[tii, . . . , u^] of certain homogeneous elements ui, . . . , n^, 

def 

where i = dim V^. Throughout the paper we will call such ui, . . . ,Ui homogeneous generators or, 
collectively, a homogeneous generating set of S{V)^ . The homogeneous generators ui, . . . ,Ui 
are not unique, but their degrees di, . . . ,d£ (which we traditionally list in the increasing order) 
are uniquely defined for each group W; they are called the exponents of the group. In particular, 

di = 2 iff = {0}; the largest exponent h '= di is called the Coxeter number of W. The 



monomials u°- '= u^^ ■ ■ ■ u^^ where ai, . . . , G Z>o form an additive basis in S{V) 

Let -< be the inverse lexicographic order on Z>q: for a, a' G Z>q we write a' ^ o if the last 
non-zero coordinate of the vector a — a' is positive. The following is our first result asserting 
the existence and uniqueness of canonical invariants: 

Theorem 1.1 (Canonical invariants). Suppose that the degrees di, . . . ,di are all distinct. Then 
for each a = (oi, . . . , a^) G Z>q there exists a homogeneous element ha = h^a^ G Sc{V)^ = C{c)® 
SiyY^ unique up to multiplication by a complex constant and such that for any homogeneous 
generating set ui, . . . ,ue of S{V)^ one has: 

1. 6,GCX-n- + E.MaC(c)-n-'; 

2. (u*^ , ba)c = whenever a' -< a. 

We will prove Theorem 11.11 in Section 13.11 We will refer to each element ba as a canonical 
W -invariant in SciV)^ and to the set B = {ba \ a G Z>q}, as the canonical basis of Sc{V)^ . 
By the construction, the canonical basis B is orthogonal with respect to the form (jl.ip . 

Remark 1.1. We can extend the theorem to the case when = d^+i for some k. If V is 
irreducible, then this happens only when W is of type with even i and k = i/2. In this 
case V = X]i=i C • the positive roots are of the form Xi it Xj, and let a : V ^ V he the 
, . ] Xi if i < i 

involution given by cr[Xi) = < , i.e., a is acting on roots as the symmetry of the 

\—X£ if i = i 

Dynkin diagram. Then Theorem 11.11 holds verbatim for any choice of homogeneous generators 
ui, . . . ,U£ of S{V)^ such that o"(n^/2) = ~U£/2 (i-e., U£/2 G C • xi • • • X£) and a{uj) = uj for all 

An equality dfc = d^j^x can also happen when V is reducible, i.e., V = Vi ® V2, W = Wi x W2 
and each Wi is a reflection group of Vi. This case can be handled by induction because S{V)^ = 



w 
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Remark 1.2. Theorem 11.11 generalizes to all complex reflection groups if one replaces the 
symmetric bilinear form on V with the Hermitian one that canonically extends the M^-invariant 
Hermitian form on Sc{V) (provided that c{s~^) = c{s) for all complex reflections s). The case of 
equal degrees dk can be treated along the lines of Remark 11.11 More precisely, the phenomenon 
dk = dk+i occurs only for the following irreducible complex reflection groups (see e.g., [HIS]): 

1. The series G{m,p,£) with £ >2, p\i, p\m, and dk = d^+i = m£/p, k = I /p. 

2. The exceptional groups G7, Gn, G19 of rank 1 = 2 with di = d2 = 12, 24, 60, respectively. 

In the case [H similarly to Remark 11.11 one has V = X]i=i C • Xj, u : y — > y is the auto- 

{Xi if i < £ 
, where C, is an m-th primitive root of unity. Then 
if i = £ 

Theorem 11.11 holds verbatim for any choice of homogeneous generators ui, . . . ,U£ oi S{V)^ such 
that (T{uk) = C^^^Uk (i.e., Uk ■ {xi ■ ■ ■ Xf)'^/^) and a{ui) = Ui for all i ^ k. 

In the case [2] one can use various embeddings of rank 2 complex reflection groups (see e.g., 
[21 Section 3]) to acquire canonical invariants. For instance, G5 is a normal subgroup of index 2 
in G-j and G5 has degrees (6, 12), which implies that if {b^^^^ I '^i' '^2 £ ^>o} is the canonical 
basis for S{V)'^^, then the set {&(2ii 02) l*^!' '^2 ^ ^>o} is a (canonical) basis of S{V)'^'' . 

(c) 

Therefore, the invariants ba make sense for all complex reflection groups. 

Assume that = {0}, i.e. di = 2, and denote by L the Dunkl Laplacian Vgj : SdV) — > 
Sc{V), where 62 is the only (up to a scalar multiple) quadratic ly-invariant in S{V). Clearly, 
the restriction of L to Sc{V)^ is a well-defined linear operator Sc{V)^ —>■ Sc{V)^ . 

Proposition 1.1. Assume that = {0}. Then 

(a) For each r > the span of all &(ai,a2,.--,a^) ^^^^ ai < r is the kernel of the operator 

rr+ll 

^ l5c(V)W'- 

(5) For each a = {ai, . . . ,01) we have: 

ba = e2'6{0,a2,.-,af) 

{in particular, fe(ai,o,...,0) = ^' 
See Section 13.21 for the proof. 

The elements b(o,a2,.--,af) '-'^ canonical basis are more elusive, however we compute them 
completely when M/^ is a dihedral group. 

Theorem 1.2. Let W = l2{'m) be the dihedral group of order 2m, V = C^. 
(a) If c{si) = c{s2) = c, then the generating function of all fc(o,fc) ^-^ given by 



Y.(t)^(o,k/ = {l + emt + e^t^) 

k>0 ^ ^ 



where 62 and Cm ore elementary W -invariants {of degrees 2 and m respectively) . 

(6) If m is even and c{si) 7^0(52), then {using the notation C '= c{si)+c{s2), 6 '= c{s2)—c{si), 
^'m = i^m — ^^T^'^) '^^ have: 

E ^-^^^^^\o,u,t' = l\^-r + tr{e-'' + re'j) r~^~'dr. (1.2) 



fc>0 
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We will prove the theorem in Section 13.51 by explicitly reducing the Dunkl Laplacians to the 
Jacobi operators. In fact, it is easy to see that the formula (|1.2|) is equivalent to: 

^ fc!ef/^ (^C±l±l,^C^) f era \ 

where p'j^'^\y) is the A;-th Jacobi polynomial (see e.g. [U Section 6.3] or formula ()3.7p below). 
This and other of our arguments bear some similarity with methods of the seminal papers [8] 
and [11] where Jacobi polynomials were first studied in the context of Dunkl operators. 

Returning to the general case, note that degfta = ^d^ak- For each d € Z>q such that 

S{V)Y 7^ {0} we set e^f^ ba^^^, where Omax G ^>o is maximal with respect to -< among all 

a G Z>Q such that ^di^a^ = d. By the construction, dege^*^^ = d. The following result was 
essentially proved in our previous paper [2]. 



Theorem 1.3. Let the exponents di < ■ ■ ■ < d£ be pairwise distinct. Then 

(c) 



(a) The elements e^J^^ , . . . ,e^f^ generate the algebra SdVY^ ■ 



ih) Each e^^ is determined {up to a multiple) by its homogeneity degree d = d^ and the equation 

ih ■ 

(c) For each k = 1,2,... there is a unique, up to a multiple, element e^.*^ € Sc{V)^ {where 
h = di is the Coxeter number) of the homogeneity degree d = kh satisfying the equation 

-kh' 



\/ p{e^ ) = for any W -invariant polynomial P € SciV)^ such that degP < d^. 



Vp(e[^2) = f^^ ^''^y W -invariant polynomial P G SiV)^ such that degP < h. 



We will give a new proof of Theorem 1 1.3 1 in Section [3^21 The proof will rely on the construction 
of canonical invariants in Theorem II. 1[ 

Following [2], we refer to each e^"^^ as the canonical elementary W -invariant and each e^^j^ as 
the canonical quasiharmonic W -invariant. 

The elementary invariants for c = were, most apparently, defined by Dynkin (see e.g. [16] ) 
and later explicitly computed by K. Iwasaki in [15]. We extend the results of [15] to all c in 
Theorem 11.41 below. 

We will also construct elementary invariants for W = S^V = C" with the natural S^-action. 
It is convenient to identify SciV) with the algebra C(c)[xi, . . . , rE„] of polynomials in n variables 
depending rationally on c. The degrees dk are here dfc = A;, /c = 1, . . . , n, so Theorem 11.11 and 
Theorem 11.31 are applicable. 

To give the explicit formula for the invariants 6^ ^ define polynomials //^ ^ S C(c)[xi, . . . , 3;,^], 
/c = 2, . . . , n, by 



.(c) _ 

s=l 



where A(zi, . . . , z^.) = rii<j<j<r(-2j — Zj) is the Vandermonde determinant. Clearly, G 
C(c)[xi,...,a;„]^'=x^"-''-. 

Theorem 1.4. For all 2 < k < n there exists at^n G C(c)^ such that the k-th elementary 
canonical invariant e[f^ G C(c)[xi, . . . , x^]'^" is given by: 



^k 



weSn/iSkXSn-k) 
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We prove Theorem 1 1.41 in Section [3. 3i Our proof (as well as the formula (jl.Sp ) is very similar 
to the one by K. Iwasaki who (using dp instead of Dunkl operators Vp) computed e^^^ in |15j . 

Following his argument, one can construct the elementary canonical invariants e^^^ for other 
classical groups as well. 

Note that the formula (jl.Sp resembles the polynomial expansion of the elementary symmetric 
polynomial ek = ekixi, . . . , Xn)- 

ek= Yl w{xi---xk)= Yl ^h---^3k- 

t«e5„/(5fcX5„_fe) l<ji<-<jk<n 

The following main result demonstrates that this observation is not a mere coincidence. 

Theorem 1.5. Let W = Sn- Then for all k = 2, . . . ,n the elementary canonical invariants e^^^ 
have no poles at the singular value c = 1/n, and 

limer)=e.fx, ^ ^ ^ 



i/n \ n n 

This result allows to introduce the elementary invariant polynomials for other reflection 
groups via e^^ = lim e\ , where h is the Coxeter number. 

c—*l/h 

We will prove Theorem 11.51 in Section 13.41 by analyzing the behaviour of the form (jl.ip 
near c = 1/n. Note, however, that we could not derive the theorem directly from the explicit 
formula (jl.3p . 

Example 1.1. Denote ek{x) ek{xi — . . . ,Xn — ^^^)- It is easy to see that e^f* = 0, 
e^^^ = Cfc for = 2, 3. Direct computations for all n using (|1.3p show that 

(c) (n — 2)(n — 3) 1 — nc _2 



2n (n^ — n)c — n — 1 

(c) (n — 3)(n — 4) 1 — nc 
^ n (n^ — n)c — n — 5 

thus confirming Theorem 1 1.5[ 



4 + 64, (1.4) 
6263 + 65, (1.5) 



Remark 1.3. The definition of the canonical invariants ba and some later formulas involving 
them (e.g. (|2.2p and (|2.3p ) suggest, for W = Sn, a close relation between canonical invariants ba 
and Jack polynomials (see e.g. |19j for definition). Direct computations show, though, 
that these polynomials are not the same. \17\ equation (7)] shows, in particular, that the 
expression of via elementary symmetric polynomials 6j does not depend on n; for instance, 
J^'^^ = 6fc for all n and k (the partition contains k units). Formulas for ba, on the contrary, 
contain n explicitly (see e.g. (|1.4p l. So, the relation between ba and Jack polynomials is yet to 
be clarified. 



2 The Dunkl Laplacian and the scalar product 

Throughout the section we assume that 2 = di < ■ ■ ■ < de = h and denote 

e 

i=l 
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where xi, . . . ,X£ is any orthonormal basis in the real space V^. Obviously, 62 is a unique (up to 
a scalar multiple) quadratic VK-invariant in S'^{V). The operator L = Vgj = V^, , called the 
Dunkl Laplacian, is independent of the choice of the basis Xi ; it equals the ordinary Laplacian if 
c = 0. 

The operator L plays a key role in the theory of Dunkl operators for W. As the following 
result shows, an action of any Dunkl operator can be expressed via L: 

Lemma 2.1 ([3l equation (1.9)]). For any p G S'^iV) one has 



where p in the right-hand side means the operator of multiplication by p. 

Denote by S : Sc{V) Sc{V) the Euler vector field given by £{f) = Nf for any / G S^{V). 
Also denote he '= | J2ses (™ particular, if all c(s) are equal to a single c, then he = he). 

Proposition 2.1 ([H]). The operator E of multiplication by 62, Dunkl Laplacian L, and the 
operator H =^ 2i{l — he) + ^£ form a representation of sl2, that is, 

[E, L] = H, [H, E] = 2E, [H, L] = -2L. 
In particular, 

[L, E''] = 4kE^-\i{l - he)/2 + k-l + £) 
for all k>0. 

Denote Ud = U^^^ =^KerL n Si{V) = {/ G Si{V) \ L{f) = 0}. 



Lemma 2.2. One has 



SciV)= e^C/(^^ (2.1) 



where the direct summands are orthogonal with respect to {■,-)e- In particular, the restriction of 
(•, ■)e to each 62 • Ud is nondegenerate. 

Proof. First, note that Se{V) is an s[2-module, locally finite with respect to L, and (Bd>oUjf^ 
is the highest weight space, so that decomposition (12. ip takes place. 

Furthermore, note that the operator H from Proposition 12.11 is scalar on the space of poly- 
nomials of any given degree and therefore self-adjoint; the operators L and E are adjoint to one 
another with respect to (•, ■)c- Therefore, for ki < k2, di,d2 > one has 

(4^ • • U^f)^ = {E'^{u!,:^),E^^iU^f)), = {L'^-E'^{U:t^),uif)e 

= Sk,M-{U^f,U^f)e = ^k,MSd,42-<C{c). 

This proves the orthogonality of the decomposition. In particular, this implies that the restric- 
tion of the nondegenerate form (•, ■)e to each • Ud is nondegenerate. The lemma is proved. ■ 

Using this, we compute the form (•,-)c as follows. Denote by ipe a (unique) linear function 
Se{V) C(c) such that: 

• Mfe2) = ^cif) for ah / G Se{V)- 

• ^cif) = [f]o for all / G Ker L, where [ • ]o : SciV) C(c) is the projection defined in (jl.ip . 
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Proposition 2.2. We have: 
(a) For f € e!lU^^\ g G Si+'^^{V) one has 

d+k-l 



if, 9)c = ^cUg) ■ 4"+^=^! n (^(1 - + (2-2) 

r=0 

(6) //c : S/W — > R<x/2j i^en the restriction of ipc to 5'(l^) is given by: 

Mf) = J ( , (2.3) 

where Og & is a coroot of a reflection s € S, ^}^~^ = {x G Vk I ^2(3;) = 1} is the unit 
sphere in Vs., and an element f G S{V) is identified with a polynomial on V . 

Proof. Assume first that the function c takes only negative real values and define ipc by equa- 
tion (|2.3p . Now if A; = and f,g^ \ then the result follows from [12^ Theorem 5.2.4]. 

By definition, (pc{e2f) = ^df) for any /. Now if / = e^f, g = e^'g where / G g G uj^^ 
with d d' then {f,g)c = 0, hence fdfg) = and therefore (pdfg) = 0. So taking / G e2U^^ 
and g = e25r G 'S'c'''^*^(^)) where G Ud+2k-2r, we see that 

On the other hand, decomposition (|2.ip guarantees that {f,g)c = {f,gk)c- Therefore, to 
verify (j2.2p for any k it suffices to take g = e^^ for 5 G C^j'^'*. 
Assume that A; > 0. Then Proposition 12.11 implies that 

ielf,elg)c = {E\f),E\g)), = {E^-\f),LE\g)), = {E^-\f), [L,E\g)), 
= {E^~\f)AkE^-\i{i - K)/2 + k-l + S)~g))c 
= Ak{^{l - h,)/2 + k-l + d){el~\f), e^'-g),. 

Therefore, by induction on k, 

k 



{elf, e\~g\ = (/, ~g), ■ J] 4r(^(l - h,)/2 + r-l + d), 



r=l 

which finishes the proof for c negative real. Now (|2.2p implies that for c negative real the value 
{f,g)c depends only on the product fg (provided d and k are fixed). Since {f,g)c is a rational 
function of the values of c, this holds true for all c as well - so, one can use (|2.2p to define fc 
in the general case. ■ 

The following is the main result of the section. Denote by S{V)+ the kernel of the constant 
term projection u [u\o, see (jl.ip . Define a symmetric bilinear form <I>c : S{V)+ x S{V)+ — > 

C(c) as ^ciu,v) =^ (u, f)c/(l — he). This form extends naturally to C[c] ® 'S'(V^). Define now 
the form ¥c on C[c]/(1 - he) ® S{V) taking values in C[c]/(1 - h^ by 

^ciu,v) = tt{^c{u,v)), 

where tt : C[c] C[c]/(1 — he) is the canonical projection and u,v (z Se{V) are any elements 
such that u = 7r(?i), v = tt{v). 
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Theorem 2.1. 

(a) The form <^c takes its values in C[c]. 

(b) <I>c(n, n) 7^ for any non-zero element u G IR[c]/(l — he) ®r 5(V%)+. 

(c) For any C ^(Vm)^ the restriction of to C[c]/(1 — he) ® is nondegenerate. 



Proof. Prove (a) by induction on the degree of u. Indeed, it fohows from [U Proposition 2.1] 
that for any x,y & V one has: 

{x,y)c = (1 - hc){x,y)o, 

where {x,y)Q is the (complexified) VF-invariant form on V. Therefore, $c|yxy ~ (''Oo- Fur- 
thermore, assume that ^c{u,v) G C[c] for all u,v £ S<'^{V) + . Then for any m S ^'^ilF), 
U2 G S"^2(l^)^ g where ^1 + ^2 = we have 

MU1U2,V) = ^c{u2,S/uAv)) ^ '^c{u2,C[c] S'^-'iV)) C C[C]. 



This proves (a) 



It is possible to prove [(b)] now. Let Hq =^ {c G C^/^ | /ic = 1}; it is an affine hyperplane 
in the affine space A^/^. Then C[Ho] = C[c]/(1 - h^) and R[Ho\ = M[c]/(1 - h^) are integral 
domains (if \S/W\ = 1 then Hq is a point c=l/h and M[i?o] = I^)- 

Let A C M(A'^/^) be the algebra of all real-valued rational functions on the affine space 
^s/w i-gg^iar at Hq. This algebra is local with the maximal ideal m = (1 — he), and A/m is 
isomorphic to M(//o)i the field of fractions of Hq. Finally, denote ^^^(Vi)^- = A(Sir S{V)^. 

Proposition 2.3. The naturally extended A-linear form <I>c : /^^(Vk)^ x «Sy^(T4i)+ A satisfies: 

if ^c{u,u) G (1 - he) A for some u G Sa{Vk) then u G (1 - /ic)5'^(Mr). (2.4) 

Proof. Clearly, the sl2-action from Proposition 12 . II preserves both ]R[c] (8>5(Vk) and 5*^^(1%), so 
that the orthogonal decomposition (|2.ip is valid for S'yi(MR) C Sc{V). Therefore, it suffices to 
verify ()2.4p only for u G 62?/^, where 

ild = U^"^ n SAiVu) = {ve Sa{Vu) I L{v) = 0}. 
For every such u it follows from ()2.2p that 

^ciu,u) = ipc{u^)-2A''+''~'k\ H {e{l-hc)/2 + r). (2.5) 

r=l 

Since the product in the right-hand side is not divisible by (1 — he), we see that tpciv?) G A for 
all u G ^^(Vi). Implication (j2.4p is now equivalent to the following one: 

if ipc{u^) G (1 - hc)A for some u G S'^(Vk) then u G (1 - /ic)5'yi(MR). (2.6) 

In (|2.6p . if /i = 2, i.e., = 52, we have nothing to prove. Assume that h > 2 and let Hq 
be the set of all cq : S/W lR<i/2 such that 1 — hcQ = and u has no poles at cq. By the 
very design, ^0 is a non-empty open subset of the real hyperplane Ho(K) C M'^/^. Indeed, we 
can write he = hici + • • • -|- h^Ck, where k = \S/W\ and ci, . . . ,Ck are standard coordinates on 

^S/W ^ ^k^ /j. g and /ii H ^ hj, = h. The intersection i?^ = i7o(]R) n {R<i/2)^ 

contains the point c = {1/h, ... ,1/h), hence Hq is non-empty and open in Hq{M.). The set -ffo is 
obtained from H^ by removing poles of u G A® SiVM.); so, Hq C -ffo(I^) is open and non-empty 
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as well. On the other hand, for each cq G Hq the specialization € S'(Vi) of u is well-defined. 
Therefore, our choice of u and cq implies that 

But the integral presentation ()2.3p oiipcQ guarantees that tpcoif"^) > for all nonzero polynomials 
/ € 5(1%). Hence, Ucg = 0, for all cq & Hq. If jS'/VFl = 1, i.e., Hq is a single point c = 1/h, 
then, clearly, cq = l/h and Ui/^ = implies that ii € (1 — /ic)5'yi(VM). Otherwise, if f^o is at 
least an affine line, the set i^o is infinite and is a set of regular points for a rational function 
Co I— > ■Uco- Therefore, = for all cq G -ffo(]K) and hence ii € (1 — /ic)'S'^(Mr) as well, proving 
implications (j2.6|) and ()2.4p . The proposition is proved. ■ 



Part 1(6)1 of the theorem immediately follows from Proposition 12.31 



To prove (c) we need the following obvious result: 



Lemma 2.3. Let $ : C/q x C/q — > k &e a non- degenerate symmetric bilinear form on a k-vector 
space Uq. Then for any field ¥ containing k the extension of^ to U = F0kC/o is a non- degenerate 
¥-bilinear form U x U 

We will use the lemma with k being the field of fractions of the integral domain ]R[c]/(l — he), 
¥ - the field of fractions of C[c]/(1 - h^), Uq = Ur, and $ = ¥c. It follows from part 
of the theorem that the restriction of to Uq is non-degenerate. Hence Lemma 12.31 guarantees 



the same for U = ¥ ^Uq. This proves part (c) of the theorem 



Remark 2.1. The proof of Theorem l2.]](6)| also implies unitarity of ^(Vm)^ as a module over the 
rational Cherednik algebra Hcq{W) for all cq G ^<q^ and for small cq G M^q^. This agrees with 
the results of the recent paper [13], where the unitary representations of Hc{W) were studied. 



3 Canonical basis and proofs of main results 
3.1 Proof of Theorem [TT] 

Fix a homogeneous generating set {ui, . . . , ui} of S{V)^ and take the basis u"", a G Z>0) in S{V) 
with the inverse lexicographic order. Define the C-subspaces Sc{V)^^ and Sc{V)^^ of Sc{V)^ 
by 

(clearly, Sc{V)^^ C Sc{V)^J. Note first that for each a G Z|o the spaces Sc{V)^^, Sc{V)^^ do 
not depend on the choice of generators ui, . . . ,Ui of S{V)^ . Indeed, let n'^, . . . , be another 
set of generators of S(y)'^ . Since di < d2 < ■ ■ ■ < dg, one has it^ = aiUi + Pi{ui, . . . ,Ui-i), 
where G C \ {0} and Pi is a polynomial of i — 1 variables for i = 1,2, ... ,i. 

We are going to define the canonical invariant ba G Sc{V)^^ as the unique (up to a multiple) 
vector orthogonal to the subspace Sc{V)^^. However, the uniqueness of such an element requires 
more arguments. 

Proposition 3.1. Let U be any subspace of S{V^). Then: 

(a) the restriction of the form (jl.ip to Uc = C(c) ®U is a non- degenerate symmetric bilinear 
form on Uc; 

(b) for any vector u G ^(l^) \ U there is a unique {up to a complex multiple) element b G 
C ■ u + Uc such that (6, Uc)c = 0. 
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Remark 3.1. Statement (a) of this proposition is essentially a Cherednik algebra version of the 



statements proved in statements 5.1.20 and 5.1.21] for double affine Hecke algebras. 

Proof. We need the following general fact. Let ^ be a unital commutative local ring with 
no zero-divisors, m its maximal ideal, k = A/m the residue field. In what follows we assume 
that k C w4 so that the restriction of the canonical projection ^ ^ k to k is the identity 
homomorphism k ^ k. Let U he a vector space over k and let $ : f7 x [/ ^ ^ be a k-bilinear 

dcf 

symmetric form on U ; denote hy : U x ^7 ^ k the residual form given by <l>o = vr o where 
TT : ^ — > k is the canonical quotient map. 

Lemma 3.1. In the notation as above assume that: 

1. <Poiu,u) j^O for aUueU\{0}. 

2. There exists an increasing sequence of prime ideals 

{0} = mo C mi C m2 C • • • C rrifc = m 
in A such that mj+i/mj is a principal ideal of A/xxii for i = 0, 1, . . . , A; — 1. 

dcf 

Then the natural A-bilinear extension of <& to C/_4 x U_a A, where U_a = A (Sit U , satisfies 
$(tt, tt) / for all ueUw\ {0}. 

Proof. We proceed by induction on k. For = 0, m = {0}, F = ^ = k, and we have nothing 

to prove. Assume that A; > 1. Define the quotient ring A' =^ A/mi, and m^ =^mi+i/mi in A' 
for i = 0, . . . ,k — 1. Clearly, the ring A' and its ideals m^ satisfy the assumptions of the lemma 
for k — 1; therefore, the inductive hypothesis holds in the following form: 

if <I>(tt,tt) G mi for some u G A<S>U, then n e mi ^5 f7. (3.2) 

Since the ideal mi is principal, i.e., mi = ciA, we can write each non-zero vector n S mi C/ 
in the form u = c\uq, where uq ^mi'^i) U. Therefore, the equation ^{u,u) = is equivalent to 
^{uo,uo) = 0. However, applying the inductive hypothesis (|3.2|) to any uq ^ mi ®U satisfying 
^{uo,uo) = 0, we obtain a contradiction. Therefore, ^{uo,uo) ^ for all ^ mi (g) U. Hence 
u) = for u G A ^ U ii and only if u = 0. 

The lemma is proved. ■ 

We apply the lemma in the case when F = C(c) = C(ci,...,Cfc) is the field of rational 
functions in the variables ci, . . . ,Ck (where k = \S/W\ is the number of conjugacy classes of 
reflections in W), ^ C F is the local ring of all rational functions regular at c = 0, and m^ is the 
ideal of A generated by ci, . . . , q for i = 0, 1, . . . , /c. Clearly, the ideals m^ satisfy condition [2] 
of Lemma 13. li Take U to be any subspace of ^(l^) and let ^ : U x U ^ A C M(c) be the 
restriction of the form (jl.ip to U. Since the specialization $o of $ at c = is a positive definite 
form on U, condition [1] of Lemma l3.ll holds as well. 

Thus, Lemma 13.11 guarantees that for any U C S'(Vr) each u € M(c) ® U \ {0} satisfies 
{ii,u)c / 0. 

Therefore, the restriction of the form (jl.ip to M(c) (S)U is non-degenerate. By extending the 



coefficients from ]R(c) to C(c) this immediately proves assertion (a) of Proposition [3Tl 



To prove assertion m\ denote = {u' G C(c)n + Uc \ (u' , Uc)c = 0}. Clearly, / 



and (U;r fl Uc,Uc)c = 0; hence U;r D Uc = hy assertion (a) This implies that dimU;r = 1 



Therefore, U;r = C(c) ■ b for some b G u + Uc- This completes the proof of Proposition 13. 1[ 
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Now we are ready to finish the proof of Theorem I l.li For each a G Z>q denote by 5'c(VK)-<a = 
'S'c(Mr) n Sc{V)^a (see (|3.ip ) the real forms of S{V)^a- Fix ui, . . . , to be a homogeneous gen- 
erating set of 5(Vk)^ so that (j3.ip implies that 5'c(Mr)!^ = J^a'^a^i'^)'^"' ^^'^ Sc{V^)^a = 

Therefore, Proposition 13.11 is applicable to this situation with U = SdV^)^^, u = n", 
and there exists a unique (up to a complex multiple) element b = ba Sc{V)^a such that 
{ba, Sc{V)^g) = (in particular, ba ^ Sc{V)^^). In other words, ba satisfies both conditions of 
Theorem ll.il and the theorem is proved. 



3.2 Proof of Proposition 11.11 and Theorem 11.31 



where /C is the kernel of L\g^^y^w In particular, the Hilbert series of /C is Y[ YZpT' ^° 



Proof of Proposition 11.11 To prove (a) , let ui{= e2),U2, ■ ■ ■ ,Ui be any homogeneous genera- 
ting set of S{V)^ . Theorem 11.11 guarantees that for any a = (oi, . . . , a^), a' = (a'^, . . . , a^) G Z>q 
with a'l > ai we have 

{U"' ,ba)c = 0. 

Equivalently, taking into account that (eg^^^n, 6a)c = {u, L'^~^^{ba))c for all r > 0, we obtain 
{S{V)^ , ^el~^^{ba))c = 0. Since the form (jl.ip is nondegenerate, we obtain 

L^^+^ba) = 

for all a € Z>q. This proves that = {ba | ai < r} is a (linearly independent) subset of 
the kernel of L''"*"^|^ (v)^" other hand, since 62 and L form a representation of SI2 by 

Proposition 12.11 we obtain isomorphisms of graded spaces: 

Se(y)^ = C(c)[e2]®/C, 

n 

Ker L"+%^(^)^ = (^C(c)-e^)0X: 

r=0 

e 

n 

fc=2 

dim(/C n S^{V)) = |Bo n S^{V)\ , 

hence 

dim(Ker r+^j^^^^^^,. n S^{V)) = |B, n S^{V)\ 

for all r > 0. This, together with the inclusion B^. C Ker (v)^' P'^o'^^s that B^ is a basis 

of Ker ^y ^w Part (a) is proved. 

To prove (6) , denote 

6a = e2^6(o,a2,...,af) 

for each a = {ai, . . . ,a£) G ^>o- Since ba satisfies condition [1] of Theorem ll.H to prove that 
ba = ba it suffices to verify that the elements 6a satisfy condition [2] of the same theorem. This 
is equivalent to the elements 6a being pairwise orthogonal, i.e., {ba,ba')c = whenever a / a'. 
Part (a) guarantees that both bQ^a2,...,ai and 6o^a'2,...,a^ are in the kernel of L, so by Lemma 12.21 
we obtain: 

(6a,6a')c e Sa^^a[ ' ( Va2,...,af) ' V,«2>-,<))c " ^(c) = 5a,a' " C(c) 

because the elements 6(o,a2,...,af) and 6(0^ a^) of the canonical basis B are orthogonal unless 



(02, . . . ,ai) = (a'g, . . . , a'l). This proves (6) 
Proposition 11.11 is proved. 
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Proof of Theorem 11.31 Take a homogeneous set ui,. . . ,ue of generators in S{V) , and de- 
— ^ "t^^-'-n"*. By definition, {u'^,e^^^)c = for all a G lA.^ such that 



Ofc = ■ • • = = 0. For any such non-zero a € Z>q let s < A; — 1 be the largest index such 
that / and let a' ^ a — 5s G ^>o denote 5s = (0, . . . , 1, . . . , 0), where 1 is in the s-th 
position). Then (it"', V^^ (e^'^^))c = 0. In particular, the element ^usi^^dl) Oc-orthogonal to 
all of the monomials u""' such that degn'^' = (i^ — ds- Since the form (•, •)c is nondegenerate for 
generic c, this implies ^usi^^dl) ~ ^' 

By definition e"^^ = Uk mod Sc{V)^s^. (see ()3.ip and the property[T]of the canonical basis 6a)) 
which implies that (e^'J^)'^^ . . . {e^^'^Y^ = mod Sc{V)^a- Indeed, if for every k = 1, . . . ,h 

for some /3pi,...,pj._i G C(c) then 

for some 7pi,...,p^ € C(c). Here {pi, . . . ,pt) -< (ai, . . . , a^), because the last non-zero coordinate of 
(oi — pi, . . . , —p^) is positive. Thus, the monomials {e'^^Y^ ■ • • {^^d}Y^ all ai, . . . , G Z>o 
form a basis of Sc{V)^ , and part (a) of Theorem 11.31 is proved. 

To prove part (6) let G S'c(F)^ be any element satisfying its conditions. Then for any 
a ^ 5k one has {u'',^i~^^)c = Vl\ ■ ■ ■ Vulz\{tJi{^) = 0, hence ^if^ = const • e{^^ + Y^s^^qfiqbq for 

some I3q G C(c). The element hq is homogeneous with degftg = diqi. Since di < ■ ■ ■ < dk < 
■ ■ ■ < di, one has deg5g > d^ for every q. Therefore, the homogeneity condition implies that 
H^^^ = const • e^^\ Part |(b)| is proved. 



Part (c) , given here for completeness, is proved in our previous paper [2]. 



Theorem 11.31 is proved. ■ 
3.3 Proof of Theorem [TTl 

The argument follows almost literally the original proof for c = given in [15]; we put it here 
mostly for reader's convenience. 

Abbreviate V^^. as V^; also take x =^ (xi, . . . , a;„) and V '= (Vi, . . . , V^)- 

Let c be generic. 

Lemma 3.2. A function f can be represented as (P(V))(A(x)) for some polynomial P if and 
onlyif{ek{V)){f{x))=Qfor all k. 

Proof. For c = (when Vj = dx^) it is a theorem due to Steinberg [20]. For c generic the 
Dunkl operators are conjugate to differentiations by means of some intertwining operator B 
(see [H]): Vj = B~^dx^B, and therefore efc(V)(/(x)) = is equivalent to ek{d)B{f{x)) = 0. 
By the Steinberg's theorem it means that / = B~^P(d)A = P{S/)B^^ A. But since A is skew- 
symmetric, VjA is proportional to dx^A, and therefore B~^A = AA for some constant A. The 
lemma is proved. ■ 

Consider now the polynomials fi = Vi{en^). Obviously, (efc(V))(/i) = for all k, and 
therefore fi = ((7i(V))(A(x)). Without loss of generality, gi can be taken skew-invariant with 
respect to the subgroup Gi C 5„ of permutations leaving i fixed; from degree considerations 
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one obtains (jiipc) — const • . . . , Xj, . . . , ). Euler's formula en = n ■ Y17=i^ifi implies 

now ()1.3p for k = n. 

Let Pmiui, ■ ■ ■ ) Us) '= uT + ■ ■ ■ + vT- Since p.3|) for A: = n is proved, one has 
(Pi(Vji, . . . , VjJ)(/ji,...j^(x)) = const • 5ik, 

where '\- < ji < ■ ■ ■ < jk < n and fji,...,j^. '= e^^\xj-^, . . . ,Xj^). Thus, if is the right-hand side 
of ()1.3p . one has 

Pi(V)Ffc = const- {Pii^))U3.,...,ju{^)) 

l<ji<-<jk<n 

= const- Y (Pj(Vji,---,VjJ)(/j,,...j^(x)) = const -(Jifc, 

l<ii<-<ifc<n 

Hence, = e^"'* . 

Theorem 11.41 is proved. 



3.4 Proof of Theorem ITEl 

We need the following result. 

Proposition 3.2. The canonical basis B is well-defined at he = 1 (e.g., at c= l/h) and B\{1} 
is orthogonal with respect to the form 

{see Theorem \2.1^ . 

Proof. Indeed, in the definition of ba in Theorem 1 1.1 1 we can take ui, . . . , to be a homogeneous 
generating set of S{Vm.)^ ■ Let us prove that for each a G Z^q \ {0} the coefficients Ca^a' of the 
expansion ba = Yla'<a^a,a.>u"' have no poles at he = 1. Suppose the contrary. Then there 
exists an exponent ^ > such that 6a = (1 — hcYba is regular at /ic = 1 and 7r(6a) 7^ 0, where 
IT : A ^ A/{1 — he) is the canonical homomorphism {A is as in Proposition 12.30 . Since i > 0, 

we see that 7r(6a) G U Z^a'^a a'^o u"'' , where k = A/ {I — he). But since ^c{ba, u°-') = for 
all a' -< a, we see that ^e{'^{ba),U) = which contradicts Theorem I2.1](c)[ The contradiction 
obtained proves that i = 0, i.e., 'iT{ba) is well-defined. The orthogonality of these elements is 
obvious. ■ 

Let W = Sn, V = C". Here dk = k, k = l,...,n. Recall that = ek{xi, . . . , Xn) € 
C[xi, . . . ,Xn] is the k-th elementary symmetric polynomial and = efc(xi — ^^^^ , . . . ,x„ — ^i^). 
The elements Ck, k = 2, . . . ,n generate a subalgebra of S(y)^" isomorphic to S{V')'^", where 

V' = {xev \ ZiXi = o}. 

Lemma 3.3. Using the notation Pr'=' Hi + ■ • • + Vn^ obtain 

VpM) = (-l)'(^ - A; + 1) . . . (n - A; + r) ~ "'^^^ ^ ~ ""^^ e^.,. 

n^c 

Proof. Denote 

^ . X dcf / _ ei(x) _ ei{x) _ ei{x) 

\ n n n 

(the i-th argument omitted). Easy calculations show that 

n 

^Qfc,i = (n - A;)efc, (3.3) 
1=1 
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dQk,i _ ?^ - fc 

Q — Qk—l,ii 

oxi n 

■q^ = ^y,{^k) = Qk-i,i efc_i. (3.4) 

Clearly, 

E{ij)Qk,i — Qk,i I , 
= (n - k)Qk-i,i, 

and therefore the Dunkl operator 

yyAQk,i) = (n - A;) - Qk^iA- 

An mduction on r (where ()3.4p is the base) gives then 

V;^{ek) = {n-k + l)---{n-k + r-l)^ L_^-Lq 

. (n-A; + l)---(n-fe + r) _ 

+ (-Ij ; Cfc-r- 



Summation over i using (jS.Sp finishes the proof. ■ 

Lemma [331 implies that ^^^i^k) = for c = 1/n and all r < k. Since pi, . . . generate 
C[2;i, . . . , Xn]^", we obtain 

Vj)(efc) = 

for c = 1/n and any homogeneous symmetric polynomial P of degree degP < k. 

On the other hand, by Theorem 1 1 . !^(6) | one has Vp(e^'^^) = for any homogeneous symmetric 
polynomial P of degree deg P < k. Therefore, 

¥i/„(e-',e-fc) = = ¥i/„(e-',el^/")) 

for any a' -< 6k- Here we abbreviated '= with c = 1/n in the notation of Theorem 12.11 
and e^^^"^ =^ linic^i/h which is well-defined by Proposition 13.21 Consequently, 

¥i/„(C/,el'/") -aefc) = 

for all a' ~< 6k, a G C^, where U = *^ ' Therefore, e[.^''"'^ = acfc because, on the one 

hand, e^.^^"'' — ack G f/ for some a ^ 0, and on the other hand, the restriction of ^i/n to U is 



non-degenerate by Theorem 12. ]|(c)[ 
Theorem 11.51 is proved. 

3.5 Canonical invariants of dihedral groups and proof of Theorem 11.21 

Throughout the section we deal with the dihedral group 

W = him) = {si = sl = (sosir = 1) 
of order 2m. 
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We denote by {z, z} the basis of V such that 

sj{z) = -C^z, sj{z) = -C'z. (3.5) 

for j = 0, 1, where (" = e^'^*/™' is an m-th primitive root of unity. 
We also denote by 62, Cm the generators of S{V)'^ given by 

e2 = zz, em = z^ + z'^. 

Lemma 3.4. The restriction of the Dunkl Laplacian L to S{V)^ = C[e2,em] for W = l2{'m) 
equals: 

,2 



L = 62^4 + me^de.de^ + m'e^-'dt^ + (^1 - -C) 8,, + — ^e^^'^^e 



dcf , s , , s , r dcf 



where C = c(si) +0(52) and 5 = c{s2) — c(si) {so that 5 = when m is odd); and de,^ mean 
here differentiation with respect to 62 and Cm, respectively, in the ring C[e2,em]- 

Proof. Clearly, s o dy = ds*[y) o s for any linear automorphism s of y and any y V*, where 
dy : S{V) S{V) is the directional derivative. So, L can be rewritten in the form L = ^ ■ DiSi 
where Di are differential operators (of order at most 2) and Sj are reflections. Thus, on the space 
of invariant functions L is a second order differential operator. To determine its coefficients 
it suffices to compute L on monomials of degree 1 and 2 in 62, Cm- This is done in [10]; see 
also [2]. ■ 



Corollary 3.1. In the notation of Lemma \2>A{ we have 



4 . o /.e: 



-2 



m/2 ~ 



2-62^ = £' -C£+[A^ -1\{V^ -V)+\C -1 + 26^- V 

where £ '= ^e2(?e2 + ^mdem ^-^ multiple of the Euler derivation, and D ^= Cmde^- 
Proof. Since de2e2 = e2(?e2 + 1, one obtains 

Ae2L = {2e2de,f + 4m(e29eJ(e„9e„) + Am^e':^dl^ - 2mCe2de, + 2m''5e'^'^ d^^. 
Equalities {2e2de^Y + Am{e2de^){emdeJ) = m'^{£^ - V^) and = 4-(p2 _ imply now that 

A rn 

—e2L = £^ -V'^ + 4^(p2 -V)- C{£ -V) + 25^^V, 

and the corollary follows. ■ 
Corollary 3.2 (of Corollary El]) . Let f = f{x,u) e C[[x,u]]. Then 

where 

Lcc,u = u^dl - (C - l)udu + (4 - x^)dl + {{C - l)x + 25)d^. 
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Proof. Follows from 



and 



m/2. 
m/2 ' 2 



Proof of Theorem 11.21 In view of Corollary 13.21 for c = const it suffices to prove that 
Lx,u{p'^) = 0, where p = 1 + xu + u^. Indeed, 

udu{p^) = cu{x + 2u)p''~^ = 2cp^ - c(2 + ux)p'^~^ , 
{{uduf - 2cudu){p') = uduiudu - 2cW) = -cudu{{2 + nx)/-^) 
= -cuxp''~^ - c{c - l)u{2 + ux){x + 2u)p''~'^, 

dxip"") = cup"-^, 
dl{jf) = c{c-\)u^p^-'^. 

For c = const one has C = 2c and 5 = 0, so that 

c'^p^~''Lx,u{p'') = -uxp - (c - l)u(2 + ux){x + 2u) + (4 - x^)(c - l)n^ + (2c - l)xup 
= (c - l)u{2xp - (2 + ux){x + 2u) + (4 - x'^)u) 
= 0. 



This proves part (a) of Theorem 11.21 

To prove part |(6)| define Ir'^'^\y,u) G C[[y,ti]] for each r, a, 6 by 

4'^'*) = ^'^+''(1 - s)-^-!-'" (i-s + us{l- |(1 - y)))' ds. 

Clearly, the right-hand side of ([L2D equals /^"'*^ (y, u) with a = -(C+5+l)/2, 6 = -(C-(5+l)/2, 
r = -6 - 1, n = e^/'t, and y = (so that ^-^ = -^). 

Therefore, in view of Corollary 13.2^ it suffices to prove that 

^x,n(4"''^(x/2,n)) =0 (3.6) 

for all r, where a, 6 are as above, and to determine the normalizing coefficients ^^(c). 
Recall that the n-th Jacobi polynomial pjf'^\y) is given by 

Pt'Hy) = , , ^^"f't^! TT f's-+^+'{l - fl - -(1 - y)Yds (3.7) 

nT{n + a + b + l)T{-n-b) Jo ^ ' \ 2^ ^ ' 

(with the analytic continuation to all a, 6 G C). Thus one has 

li'^'") (y, u) = [' s^+\l - s)-'-' f; r(r + l) ^ _ „ (1 _ £(1 _ y))" 

^nlFr-n + l Jo V 2^ 

n=0 
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^ r(r + l)r(n + a + b+ l)T{-n - b) M),.n 
^ r(r-n+l)r(a + n+l) " 

n=0 



oo 

dcf 



n=0 



The Jacobi polynomial p'^^'^^ (y) belongs to the kernel of the differential operator 
j(a,fe) drf (1 _ ^2)^2 + (5 _ a _ + 5 + 2)y)9y + n(n + a + 6 + 1) 

(see e.g. [1] for proof). Therefore, = '^'^^Qqn{r,a,b)PlC'''\y)u'^ satisfies 

^J:if^(4"''^(y,n)) = o, (3.8) 

where L^y,^'^ = (1 - ^2)^2 + (6 - a - (a + 6 + 2)y)dy + u^d^ + (a + 6 + 2)n5„. 

Take now a = -(C + 5+ l)/2, b = -{C -6 + l)/2, so that a + 6 + 2 = -(C- 1), 6 - a = J. 
One has then 



^J:^^ = (1 - + (5 + (c - i)y)dy + ^z^a^ - (c - i)t.a„ 

^2y 

i2y,M- 



(4 - (2y)2)a|^ + {26 +{C- l){2y))d2y + u'dl - {C - l)udu 



Therefore, imphes (ISTe]) . 

To finish the proof it remains to find the value of the normalization coefficients = nfc(c) 
in ()1.2p . To do this, substitute 62 = into (jl.2p . so that e'„^ = 6^/4. Under this specialization, 
^(0,fc) becomes a (complex) multiple of and the right-hand side of ()1.2p becomes (with the 
abbreviation a = ): 



fc=0 

CO 

E 

k=0 



a\ r{a-k + l)r(-C + 2k) ^ (e. 
r(a + A; - C + 1) 



r(a + l)r(-g + 2fc) /e^N 

^/fc!-r(a + yfc-C + l) V 4 
fc=o ^ ' 



_ - r(^)r(2fc-c) e^, 
A. r(fc_ci|^) 4'=^!- 

This proves ()1.2p and finishes the proof of part |(5)| of Theorem ll.2[ ■ 
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